Smooth Contextual Bandits: Bridging the Parametric and Nondifferentiable Regret Regimes

Dynamic Personalized Decision Making Beyond the Super-Extrapolatable and Super-Local Cases Contextual bandit problems model inherent trade-off between exploration exploitation in personalized decision making marketing, healthcare, revenue management, more. Specifically, is characterized by optimal growth rate of regret. Intuitively, should depend on how complex underlying supervised learning problem is, namely, much can observing reward one context tell us about mean rewards another. To formalize this intuitive relationship, Hu, Kallus, Mao study “Smooth Bandits: Bridging Parametric Nondifferentiable Regimes” a nonparametric contextual which functions are β-times differentiable (more generally, Hölder β-smooth). This interpolates two extremes previously studied isolation: nondifferentiable bandits (β ≤ 1), with running separated noncontextual different regions achieves rate-optimal regret, parametric-response = ∞), regret be achieved minimal or no because infinite extrapolatability across contexts. The authors develop algorithm that operates neither fully locally nor globally, revealing in-between smooth setting shedding light crucial interplay functional complexity dynamic making.